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na Thursday afternoon in April, Elena's eyes scan a computer screen at the Providence Public Library. 
As a student in the library's Rhode Coders program, she’s using HTML and JavaScript to build a video 
game modeled on PacMan. This afternoon the game isn’t working correctly so she's sifting through her 
code to find the source of her error. She’s so determined to fix her game that she can hardly hear the 
buzz around her as her peers pack up for the day. 


Nearing the end of her junior year, Elena is preparing to apply to college where she plans to study computer science. 
For years, Elena thought that college wasn’t for her. She found school boring and disconnected from her life. Her 
experience in Rhode Coders changed her perspective. In an environment where she was invited to own her learning 
experience, Elena has excelled. 


Though Rhode Coders offers computer programming instruction, developing students’ technical coding skills is not 
the program's primary goal. Instead staff seek to equip students with the real-world skills, such as perseverance and 
teamwork, that they need to succeed in college and the workplace. To elevate the importance of these skills, Rhode 
Coders joined a network of after school programs in Providence and Boston issuing credentials for skills including 
teamwork, engagement, perseverance, communication, and critical thinking through digital skills badges. Programs 
recognize that these skills are critical to students’ postgraduate success but are often undervalued by colleges 

and employers making acceptance and hiring decisions. Digital badges, which serve as portable documentation 

of students’ skill attainment, provide a means to measure students’ skill development and credit them for their 
learning. 


The badging initiative made a difference for Elena. Along with her peers, she helped Rhode Coders define what it 
means to demonstrate perseverance. Using this definition, she meets with her instructor regularly to set goals and 
discuss her progress. This process helped Elena recognize the areas where she perseveres both within and outside 


her program. She recognizes perseverance in her ability to remain 
focused on her college goals despite her fears about paying for high- 
er education and her uncertainty about being the first in her family 
to go to college. She now has the language to articulate this strength 
in job interviews and college applications. When asked to provide 
evidence of her skills, Elena can refer to her digital badge. 


For the past two years, the Rennie Center followed teams in Provi- 
dence and Boston as they engaged in a bold experiment to creden- 
tial the skills that young people, like Elena, gain outside of school. 
Though strengthening real-world skills has been a longstanding goal 
of many after-school programs, creating a competency-based cre- 
dentialing system was a complex task. The following report shares 
successes and lessons learned from this work, which was led by 
Boston After School & Beyond (Boston Beyond) and the Providence 
After School Alliance (PASA). These lessons learned are designed to 
provide guidance for student-serving organizations exploring oppor- 
tunities to formally recognize students’ nonacademic skills. 


The Leaders: Innovators in 
Out of School Time Learning 


About Boston After School & Beyond and the Providence 
After School Alliance 

In their respective cities, Boston Beyond and PASA are intermediaries 
that coordinate networks of after-school and summer learning op- 
portunities, expanding access to high-quality programming for urban 
youth. They have long recognized that learning does not end at the 
conclusion of the school day or year. Through music, arts, athletics, 
and leadership programs, youth explore the future that they want to 
create for themselves. They gain valuable skills, such as the ability to 
think critically or work in a team, that support them in reaching their 
goals. 


PASA and Boston Beyond recognize that when privileged students 
complete job and college applications, they often showcase their 
non-academic skills via volunteer experience, unpaid internships, or 
music lessons. This further disadvantages marginalized students who 
have not traditionally had access to this type of experiential learn- 
ing. Leaders at Boston Beyond and PASA are determined to change 
this. By strengthening the quality of out-of-school learning and using 
digital badges to showcase a holistic view of students’ skills, Boston 
Beyond and PASA seek to unlock opportunity for youth. 


. Student enrolls in an out-of- 
school-time program. 


. Instructor introduces 


competencies associated 
with the program. Students 
partner with instructors to 
define each skill. Examples: 
Perseverance, Critical 
Thinking, Teamwork 


. Instructor introduces 


program activities and the 
measurement process, 
which encourages students 
to develop an improvement 
mindset. 
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measure skill attainment 
and growth throughout the 
program and document 
scores in an online badging 
platform. 


. If students meet the criteria, 


the platform automatically 
triggers the award of a 
digital badge. 


. Students upload additional 


evidence of skill attainment 
to be stored in the badge. 


. Students log into the 


badging system to access 
the badge, which they can 
post on LinkedIn, social 
media, or other sites. 
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What is a Digital Skill Badge? 


A digital badge is a visual representation of a skill that a student possesses. Badges can be posted on resumes, job 
applications, college applications, and networking sites including Facebook and LinkedIn. They are grounded in verifi- 
able evidence, showing what the badge recipient did to earn the credential. By clicking on the badge icon, individuals 
can learn about the student's work and the organization that awarded the badge. 


The criteria for students to earn a badge are unique to each city. A brief description of the Boston and Providence 
processes are included below. 


Providence Digital Badges 

In Providence, students earn badges based on a rubric which defines the sub-elements of each skill and the specific 
ways in which students practice the skill within their program. Each youth sets individual growth goals to focus their 
learning within each skill. Youth and instructors monitor progress regularly. In order to earn a badge, students must 
demonstrate exemplary performance in each sub-element of the skill and provide evidence of their learning. 


Boston Digital Badges 

In Boston, students earn badges based on a research-validated skills assessment, 80% attendance, and a portfolio 
that demonstrates the work they completed to attain the skill. At the beginning of the program, students define their 
chosen skill(s) and set goals to improve their abilities. As the program progresses, instructors facilitate self-reflection, 
peer feedback, and teacher evaluation sessions for students to assess their skill development. At the end of the pro- 
gram, students demonstrate their skill gains and engage in a final reflection on their experience via their portfolio. 


ed 


CRITICAL THINKING 


PERSEVERANCE COMMUNICATION 


The Online Platform 


Leaders in Providence and Boston agree that their work would not have been possible without an online tool that 
connects programs, allows instructors to measure skill growth, and administers badges. Both cities use ForAllSys- 
tems, a tool with a student-facing component that allows youth to store, manage, and share their badges. The tool 
also includes an administrative view to facilitate system-wide coordination between the intermediary and the 
program instructors. 
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What We've Learned 


In building a digital badging system, Boston and Providence team leads faced two competing priorities. Consisten- 

cy across programs was important to gaining recognition of digital badges amongst colleges, school districts, and 
employers. At the same time, programs needed flexibility to allow youth to own their skill-building process. Over 

the course of two years, PASA and Boston Beyond created a competency-based system that awarded students for 
attaining a skill rather than for completing a certain number of program hours. This system was designed to provide a 
consistent structure that could be implemented flexibly across diverse programs. Lessons learned from this effort are 
highlighted in this section. 


>» Students Need to Own their Skills (and their Badges) 


For Mia, a student at Providence’s Young Voices, perseverance was something that adults always told her she needed 
to do. When she became bored in math class or didn’t want to finish a book, her teachers told her to persevere. It 
wasn't until Mia joined Young Voices that she defined perseverance for herself. 


As a girl of color in Providence schools, Mia saw firsthand the disparities in discipline between herself and her white 
peers. She saw other students of color sent home from school for lingering in the hallways or wearing clothing the 
school deemed inappropriate, while white students were not punished for the same behavior. In Young Voices, she 
learned to use her voice to address problems in her school community. She joined a group of students advocating for 
change by speaking up to the Providence City Council and School Board. Perseverance took on new meaning for her. 
“| learned that being outspoken is not an annoyance. It’s a necessity,” Mia said. “Any issue that’s been fixed is through 
people speaking out and advocating.” 


Mia is proud of the digital badge she earned 
for perseverance. It reminds her of the work 
she did on behalf of herself and her peers 

in Providence. Her experience illustrates the 
importance of youth ownership in the badging 
process. Too often in school, students are told 
where and when they must develop a skill. In 
Providence, leaders recognized that their role is 
to create opportunities for students to develop 
skills and give students the language to articu- 
late these skills. “There’s a point where adults 
need to get out of the way,” one instructor said. 
“The young people can tell us what matters to 
them and what they want to learn.” 


KEY TAKEAWAY 


Digital badging must focus on skills that 
matter to students. Programs should provide 
opportunities for students to demonstrate 
these skills in personally meaningful ways. 


>» Use Assessment as 
a Tool for Youth and 
Adult Learning 

For students like Mia, owning the work includes having a voice in the way they are assessed. Too often, she said, as- 

sessments don't help students take control of their learning. In Boston, programs were determined to change this by 

using assessment as a youth-led learning tool. 


In the first year of the pilot, Boston Beyond worked with individual programs to create their own rubrics to measure 
skill attainment. Though this provided the opportunity for programs to authentically measure competencies, they 
quickly learned that this approach would not scale. Programs did not have the internal capacity to create a measure- 
ment system for each skill, and standards for skill attainment varied across programs. 


In year two, Boston Beyond shifted their practice based on feedback from program instructors. Instead of using pro- 
gram-specific rubrics to measure skill attainment, they created a three-part measurement process. In order to earn 
a badge under the new system, students need to earn a score of four out of five on a validated, research-based skills 
assessment called the Survey of Academic and Youth Outcomes (SAYO). They also need to attend the program at 
least 80% of the time and create a portfolio demonstrating evidence of skill attainment. 


When digital badging initially launched, only adults had access to SAYO data. To create student ownership in the 
assessment process, instructors began to share SAYO data with students and used this data to have honest conversa- 
tions about student progress. Many students used these conversations to advocate for the support they needed from 
instructors to reach their goals. The portfolio created an additional opportunity to shift the role of the instructor, as it 
gave students the opportunity to choose how they would demonstrate their learning. Instructors became facilitators 
of this process, offering guidance while allowing students to lead. 
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Instructors acknowledged that student ownership of the assessment process remains varied. While many instructors 
are eager to provide opportunities for student ownership, some have struggled to cede power to students. Leaders 
remain hopeful that instructors struggling with this mindset shift will learn from their peers who have seen success 


under the new model. 


KEY TAKEAWAY 
Provide opportunities for student ownership in the assessment process. Foster an environment in which adults 
welcome youth feedback and adapt programming accordingly. 


>» Make Learning Transparent 


At the beginning of the pilot, the assessment process for digital badges was so different from assessments in school 
that some students and instructors were skeptical. They wondered whether colleges and businesses who are accus- 
tomed to letter grades and test scores would ever accept a new credential. Given that the badging pilot focused on 
lifelong skills, they recognized a need to define what proficiency looked like in each skill area. They knew that badges 
would only gain currency if those viewing students’ resumes clearly understood what the badge represents. 


Despite general agreement on skill definitions, instructors in Providence recognize that skills look different across 
programs. For Elena in Rhode Coders, effective communication included her ability to explain the error that she iden- 
tified in her code. For Mia in Young Voices, effective communication included her ability to create a specific message 
to share with city councilors and school board members. The PASA team recognized this necessity by including pro- 
gram-specific criteria in the rubric. 


An example of program-specific criteria from Youth in Action (YIA), a youth leadership and advocacy organization, 
is included below. The organization provides youth with opportunities to facilitate workshops for peers and adults. 


These youth-led workshops frequently involve difficult conversations on topics including community health and 
student-centered learning. In creating their communication criteria, YIA staff and students reflected on the skills that 
youth need to facilitate engaging, action-oriented workshops. 


Providence students see the transparency of their rubric as an important step towards digital badge currency. When 
a student earns an A on a traditional academic transcript, those reviewing the transcript have no information about 
the student's work. When Mia shares her digital badge from Young Voices with potential employers, it provides evi- 
dence that she can effectively communicate with diverse audiences, facilitate difficult conversations, and read others’ 
verbal and nonverbal communication. 


Oral: The student communicates e Student makes their own conclusions about power 
points clearly, is articulate, rec- structures and the play between identity and society 
ognizes audience and adjusts as and is able to clearly communicate and articulate that 
needed; uses appropriate tone of conclusion 


voice; uses appropriate timing and 


; e Student understands the difference between presentation 
pace of speaking 


and facilitation 


Written: Student writing is well *Student is able to read a room effectively 
organized and sequential; express- 
es complete thoughts Student is able to guide a conversation to desired results 


Student designs, authors, and catalogues workshops 
Non-Verbal: Student body posture : 6 6 P 


and gestures indicate engagement. Student creates welcoming spaces for others to engage in 
Student uses eye contact; respects 


physical space; indicates aware- 
ness of facial expressions. e Student is able to convey information to various groups 


and people based on age, background, and craft 


e Student challenges those around them respectfully 


Though the measurement process differs, transparent learning is a priority among Boston programs as well. In addi- 
tion to sharing portfolios within the digital badge, programs equip students with the language to speak to their skills 
and their learning. Using digital badges as evidence, students are learning to share their accomplishments during 
interviews, on their resumes, and in application essays. Though youth still envision a future when a digital badge will 
be weighted similarly to a test score or course grade, they recognize that significant work remains. By sharing their 
experiences in conversations with colleges and employers, youth hope to awaken adults to the importance of their 
out-of-school learning. 


KEY TAKEAWAY 


Badges should contain program-specific information about the work that students completed to earn the 
badge. Given that digital badges are not yet widely recognized, youth need the language to advocate for their 
accomplishments. 
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>» Evidence of Success 


After a two-year pilot, programs in Provi- 
dence and Boston found that badging led 

to program-, instructor-, and student-level 
improvements. Within both cities, programs 
strengthened their ability to spread effective 
practices. Instructors became more intention- 
al about the skill development opportunities 
they provide to students. Students learned to 
articulate how their skills will support their fu- 
ture endeavors. The examples below highlight 
strengths noted by students, instructors and 
program leads participating in the pilot. 


Program-Level: Citywide Connections 
For program leads and instructors, digital 
badging provided a new opportunity to work 
with other youth development organizations. 
In the second year of the pilot, PASA and Bos- 


‘ NAS , 


ton Beyond launched instructor professional development with an emphasis on peer-to-peer learning. They sought 


feedback from instructors about the type of training they needed to feel comfortable with digital badging. Instructors 
spoke of a desire to deepen their own understandings of each skill and to better engage youth in the measurement 
process. 


At the beginning of the 2019-2020 school year, Boston instructors met to share successes and solve common prob- 
lems. Those who struggled to integrate youth ownership learned strategies from other programs. In Providence, 
instructors who demonstrated promising practices in the 2018-2019 school year were invited to share with the larger 
group through a yearlong community of practice. Instructors were energized by the opportunity to learn from their 
colleagues. PASA and Boston Beyond leads have begun to see effective practice spreading across programs. 


Instructor-Level: Positive Relationships 

Instructors found that the work led to unexpected improvements in their relationships with students. For Jeremiah, 
an instructor at a Providence program, monitoring progress at the skill level changed the way he looks at his students. 
He gave the example of Anthony, a bright teenager who is easily distracted by other students. Rather than focusing 
on Anthony's challenges, the badging process gave Jeremiah an opportunity to reflect on Anthony's strengths. Jeremi- 
ah noticed Anthony's ability to share his perspective and communicate effectively with peers and adults. Jeremiah’s 
relationship with Anthony has improved and made him a more effective instructor. 


Student-Level: Mindset Shifts 


At Harlem Lacrosse in Boston, students reported that monitoring goals led to an improvement mindset. Imani, a 6th 
grader, worked with her coach to set a teamwork goal. She discusses her progress with her coach regularly. “| always 
thought teamwork was something you were good at or you weren't, but Coach Maddie said she saw me improving,” 

Imani said. For students like Imani, growth mindset transfers beyond teamwork to other areas of their lives. Imani is 

trying to improve her grades, especially in language arts where she has struggled in the past. “Coach Maddie is tutor- 
ing me...| think I'll be able to get into a great high school,” Imani said. 
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Conclusion: The Way Forward 

PASA and Boston Beyond began their digital badging efforts at a time when youth-serving organizations across the 
nation sought opportunities to recognize the real-world skills that students need for success in college, the workplace, 
and in life. In the intervening years, additional research has demonstrated the critical nature of these skills, though 
they remain undervalued in formal college and job application processes. Badging work in Providence and Boston of- 
fers a promising model for youth-serving organizations that seek to recognize and promote students’ real-world skills. 


As their work progresses, networks in both cities plan to provide youth with opportunities to shape the future of 
digital badging. In strengthening existing badging processes and developing currency for digital badges, programs 
recognize that youth have valuable information to share. To meaningfully integrate youth voice, organizations are 
learning from programs like the Boston Student Advisory Council (BSAC). The badging initiative at BSAC, a student 
leadership organization, has been youth-driven from the outset. BSAC students recently designed a digital badging 
curriculum focused on communication and perseverance, skills that are critical for youth advocates seeking to spark 
change in the community. For Jayden, a BSAC senior, the case for digital badging is clear. When someone looks at his 
resume, he doesn't care if they know what grade he received in 10" grade English. He wants them to know that he’s 
a changemaker who isn't afraid of a challenging conversation. “I want the badge to give you a sense that I’m doing 
something for a greater cause,” he said. 
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About the Rennie Center 

The Rennie Center for Education Research & Policy’s mission is to improve public education through well-in- 
formed decision-making based on deep knowledge and evidence of effective policymaking and practice. 

As Massachusetts’ preeminent voice in public education reform, we create open spaces for educators and 
policymakers to consider evidence, discuss cutting-edge issues, and develop new approaches to advance 
student learning and achievement. Through our staunch commitment to independent, non-partisan re- 
search and constructive conversations, we work to promote an education system that provides every child 
with the opportunity to be successful in school and in life. 
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